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Abstract. Patience Sorting is a combinatorial algorithm that can be viewed as an iterated, non-recursive 
form of the Schensted Insertion Algorithm. In recent work the authors have shown that Patience Sorting 
provides an algorithmic description for permutations avoiding the barred (generalized) permutation pattern 
3—1—42. Motivated by this and a recently formulated geometric form for Patience Sorting in terms of certain 
intersecting lattice paths, we study the related themes of restricted input and avoidance of similar barred 
permutation patterns. One such result is to characterize those permutations for which Patience Sorting is 
an invertible algorithm as the set of permutations simultaneously avoiding the barred patterns 3—1—42 and 
3—1—24. We then enumerate this avoidance set, which involves convolved Fibonacci numbers. 


Resume. Patience Sorting est un algorithme combinatoire que l’on peut comprendre comme etant une 
version iteree, non-recursive de la correspondence de Schensted. Dans leur travail recent les auteurs ont 
demontre que Patience Sorting donne une description algorithmique des permutations evitant le motif 
barre (generalise) 3—1—42. Motives par ceci et par une forme recemment formulee de Patience Sorting en 
termes de certaine parcours du treillis intersect ants, nous etudions les themes connexe d’input restreinte et 
permutations qui evitent de similaire motifs barres. Un de nos resultats est de caracteriser les permutations 
pour lesquelles Patience Sorting est un algorithme inversible comme etant l’ensemble des permutations 
evitant simultanement les motifs barres 3—1—42 and 3—1—24. Nous enumerons ensuite cet ensemble, qui 
utilise des convolutions des nombres de Fibonacci. 


1. Introduction 

The term Patience Sorting was introduced in 1962 by C. L. Mallows m m while studying a card 
sorting algorithm invented by A. S. C. Ross. Given a shuffled deck of cards a = c±C 2 • ■ ■ c n (which we take 
to be a permutation a £ & n ), Ross proposed the following algorithm: 

Step 1 Use what Mallows called a “patience sorting procedure” to form the subsequences 
ri,r 2 , ■ ■ ■ ,r m of a (called piles) as follows: 

• Place the first card ci from the deck into a pile r\ by itself. 

• For each remaining card Cj (i = 2,..., n), consider the cards d\,d 2 ,..., dk atop 
the piles r±, r 2 , ■ ■ ■, rk that have already been formed. 

— If Ci > maxjdi, d 2 , ■ . ■, dk}, then put Ci into a new right-most pile ru+i 
by itself. 

— Otherwise, find the left-most card dj that is larger than c.j and put the 
card Ci atop pile Tj. 
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Step 2 Gather the cards up one at a time from these piles in ascending order. 


We call Step 1 of the above algorithm Patience Sorting and denote by R(cr) = {ri, V 2 , ■ ■ ■, r m } the pile 
configuration associated to the permutation a £ 6„. Moreover, given any pile configuration R , one forms its 
reverse patience word RPW(R) by listing the piles in R “from bottom to top, left to right” (i.e., by reversing 
the so-called “far-eastern reading”). In [5j these words are characterized as being exactly the elements of 
the avoidance set SVj (3-1-42). That is, they are permutation avoiding the generalized pattern 2-31 unless 
every occurrence of 2-31 is contained within an occurrence of the generalized pattern 3-1-42. (A review of 
generalized permutation patterns can be found in Section ll. 21 below!. 

We illustrate the formation of R(cr) and RPW(R) in the following example. 


Example 1.1. Let cr = 64518723 £ ©s- Then we form the pile configuration R(a) as follows: 
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Then, by reading up the columns of R(<r) from left to right, RPW(R(a)) = 64152873 £ Sg (3-1-42). 


Given a £ ©„, the formation of R(cr) can be viewed as an iterated, non-recursive form of the Schensted 
Insertion Algorithm for interposing values into the rows of a standard Young tableau (see .2J)- In [§] the 
authors augment the formation of R(cr) so that the resulting extension of Patience Sorting becomes a full 
non-recursive analogue of the celebrated Robinson-Schensted-Knuth (or RSK) Correspondence. As with 
RSK, this Extended Patience Sorting Algorithm (given as Algorithm 11.21 in Section 11.11 below) takes a 
simple idea — that of placing cards into piles — and uses it to build a bijection between elements of the 
symmetric group ©„ and certain pairs of combinatorial objects. In the case of RSK, one uses the Schensted 
Insertion Algorithm to build a bijection with (unrestricted) pairs of standard Young tableau having the same 
shape (see ESI) However, in the case of Patience Sorting, one achieves a bijection between permutations and 
(somewhat more restricted) pairs of pile configurations having the same shape. We denote this latter bijection 
PS 

by a <—> (R(a), S(a)) and call R(a) (resp. S(a)) the insertion piles (resp. recording piles) corresponding 
to a. Collectively, we also call (R(cr), S(a)) the stable pair of pile configurations corresponding to a and 
characterize such pairs in JS] using a somewhat involved pattern avoidance condition on their reverse patience 
words. 

Barred (generalized) permutation patterns like 3-1-42 arise quite naturally when studying Patience 
Sorting. We discuss and enumerate the avoidance classes for several related patterns in Section Q Then, 
in Section |3 we examine properties of Patience Sorting under restricted input that can be characterized 
using such patterns. One such characterization, discussed in Section rm is for the crossings in the initial 
iteration of the Geometric Patience Sorting Algorithm given by the authors in [5]. This geometric form for 
the Extended Patience Sorting Algorithm is naturally dual to Viennot’s Geometric RSK (originally defined 
in HU) and gives, among other things, a geometric interpretation for the stable pairs of 3-1-42-avoiding 
permutations corresponding to a permutation under Extended Patience Sorting. However, unlike Viennot’s 
geometric form for RSK, the shadow lines in Geometric Patience Sorting are allowed to cross. While a 
complete characterization for these crossings is given in |H] in terms of the pile configurations formed, this 
new result is the first step in providing a characterization for the permutations involved in terms of barred 
pattern avoidance. 
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We close this introduction by describing both the Extended and Geometric Patience Sorting Algorithms. 
We also briefly review the notation of generalized permutation patterns. 

1.1. Extended and Geometric Patience Sorting. Mallows’ original “patience sorting procedure” 
can be extended to a full bijection between the symmetric group & n and certain restricted pairs of pile 
configurations using the following algorithm (which was first introduced in [5]1: 

Algorithm 1.2 (Extended Patience Sorting Algorithm). Given a = C 1 C 2 ■ ■ ■ c n £ & n , inductively build 
insertion piles R(cr) = {rr, r^, ■ ■ ■, r m } and recording piles S(a) = {si, s%, ■ ■ ■, s m } as follows: 

• Place the first card C\ from the deck into a pile 77 by itself, and set si = {1}. 

• For each remaining card Cj (i = 2,..., n), consider the cards di, da, • • •, dk atop the piles ri, r 2 ,..., r^ 
that have already been formed. 

— If Cj > max{di, ..., dk}, then put Cj into a new pile rk+i by itself and set Sk+ i = {*}■ 

— Otherwise, find the left-most card dj that is larger than c* and put the card Ci atop 
pile rj while simultaneously putting i at the bottom of pile Sj. 

Note that the pile configurations that comprise a resulting stable pair must have the same “shape”, which 
we define as follows: 

Definition 1.3. Given a pile configuration R = {rr, r-z, ■ ■ ■, r m } on n cards, we call the composition 
7 = (M, |r* 2 1, • ■ ■, |r m |) of n the shape of R and denote this by sh(R) =7 \= n. 

The idea behind Algorithm 11.21 is that we are using the auxiliary pile configuration S(a) to implicitly 
label the order in which the elements of the permutation <7 £ & n are added to the usual Patience Sorting 
pile configuration R(a) (which we now call the “insertion piles” of er in this context by analogy to RSK). It 
is clear that this information then allows us to uniquely reconstruct a by reversing the order in which the 
cards were played. As with normal Patience Sorting, we visualize the pile configurations R(cr) and S(a) by 
listing their constituent piles vertically as illustrated in the following example. 

Example 1.4. Given er = 64518723 £ Gg from Example o above, we simultaneously form the fol¬ 
lowing pile configurations with shape sh (R(a)) = sh(£(cr)) = (3,2,3) under Extended Patience Sorting 
(Algorithm 11 . 21 ) : 

13 15 

R(a) = 4 2 7 and S(a) =236 
6 5 8 4 7 8 


Note that the insertion piles R(cr) are the same as the pile configuration formed in Example 11.11 and that 
RPW{S{ 64518723)) = 42173865 £ S 8 (3-1-42). 

In order to now describe a natural geometric form for this Extended Patience Sorting Algorithm, we 
begin with the following fundamental definition. 

Definition 1.5. Given a lattice point (m, n) £ Z 2 , we define the (southwest) shadow of ( m,n ) to be 
the quarter space U(rn,n ) = {(x,y) £ R 2 \ x < m, y <n}. 

As with the northeasterly-oriented shadows that Viennot used when building his geometric form for 
RSK (see 1191 1. the most important use of these southwesterly-oriented shadows is in building shadowlines 
(which is illustrated in Figure llTa)): 

Definition 1.6. The (southwest) shadowline of (mi, ni), (m 2 , 712 ), ■.., (m*,, nk) £ Z 2 is defined to be 
the boundary of the union of the shadows U(m\,n\), U(rri 2 , 112 ),..., U (m^, rik)- 
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(a) Shadowline 
4 0) (64518723). 


(b) Shadowline 
4 0) (64518723). 


(c) Shadowline 
4 0) (64518723). 





(d) Salient points (©) 
for £>(°)(64518723). 


(e) Shadow Diagram 
£>« (64518723). 


(f) Shadow Diagram 
(64518723). 


Figure 1: Examples of Shadowline and Shadow Diagram Construction. 


In particular, we wish to associate to each permutation a certain collection of (southwest) shadow¬ 
lines called its shadow diagram. However, unlike the northeasterly-oriented shadowlines used to define the 
northeast shadow diagrams of Geometric RSK |l9j , these southwest shadowlines are allowed to intersect as 
illustrated in Figure ^d)-(e). (We characterize those permutations having intersecting shadowlines under 
Definition ITT! in Theorem EZD below.) 

Definition 1.7. The (southwest) shadow diagram D^ Q \a) of a = o \02 - • • o n € & n consists of the 
(southwest) shadowlines D^(a) = {L^\a), L^\a ),..., L^\a)} formed as follows: 

• Lf\a) is the shadowline for those lattice points (x, y) € {(1, or), (2, 0 - 2 ), ... ,(n, a n )} such that the 
shadow U(x,y) does not contain any other lattice points. 

• While at least one of the points (1, or), (2, 0 - 2 ),. .., (n, a n ) is not contained in the shadowlines 
Lf\a ), L^icr ),..., L^\a ), define L^ ± (a) to be the shadowline for the points 

(x, y) e A := {( i,ai ) | (i,cr*) i (J 4° } (cr)} 

fc=1 

such that the shadow U(x,y) does not contain any other lattice points from the set A. 

In other words, we define a shadow diagram by inductively eliminating points in the permutation diagram 
until every point has been used to define a shadowline (as illustrated in Figure |l|a)-(c)). 

One can prove (see :5J) that the ordinates (i.e., y-coordinates) of the points used to define each shadowline 
in the shadow diagram D^°\c r) are exactly the left-to-right minima subsequences (a.k.a. basic subsequences) 
in the permutation a £ & n - These are defined as follows: 
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Definition 1.8. Let tv = tv\tvi---t n be a partial permutation on the set [n] = {1,2,...,n.}. Then 
the left-to-right minima (resp. maxima) subsequence of tv consists of those components TVj of tv such that 
nj = min{7Tj | 1 < * < j} (resp. tV j = max{7r,; | 1 < i < j}). 

We then inductively define the left-to-right minima (resp. maxima) subsequences Si,S2,...,Sfc of the per¬ 
mutation a by taking S! to be the left-to-right minima (resp. maxima) subsequence for a itself and then 
each remaining subsequence s.j to be the left-to-right minima (resp. maxima) subsequence for the partial 
permutation obtained by removing the elements of Si, S 2 ,..., Si-± from a. 

Finally, one can produce a sequence D(a) = (D(°\cr), D«( a), D^ 2 \a), ...) of shadow diagrams for 
a given permutation a £ 6„ by recursively applying Definition 11.71 to the southwest corners (called salient 
points) of a given set of shadowlines (as illustrated in Figurenid)-(f)). The only difference is that, with each 
iteration, newly formed shadowlines can only connect salient points along the same pre-existing shadowline. 
One can then uniquely reconstruct the pile configurations R(cr) and S(a) from these shadowlines by taking 
their intersections with the x - and y-axes in a certain canonical order (as detailed in jg]). 

Definition 1.9. We call D^(cr) the fc th iterate of the exhaustive shadow diagram D(a) for the permu¬ 
tation a £ & n . 

1.2. Generalized Pattern Avoidance. We first recall the following definition: 

Definition 1.10. Let a = ay 02 • • • o n £ & n and tv £ 6 m with m < n. Then we say that a con¬ 
tains the (classical) permutation pattern tv if there exists a subsequence (cy,, oy 2 ,..., ay m ) of a (meaning 
ii < *2 < • • • < im) such that the word cr^a^ ... ay m is order-isomorphic to 7r. I.e., each ay f < <Ji j+1 if 
and only if 7 Vj < TVj+i. 

Note, though, that the elements in the subsequence (ay,, ay 2 ,..., ay m ) are not required to be contiguous 
in a. This motivates the 


Definition 1.11. A generalized permutation pattern is a classical permutation pattern 7r in which one 
assumes that every element in the subsequence (oy,, ay 2 ,..., oy m ) of a must be taken contiguously unless a 
dash is inserted between the corresponding order-isomorphic elements of the pattern tv. 


Finally, if a does not contain a subsequence that is order-isomorphic to 7r, then we say that a avoids the 
pattern tv. This motivated the 


Definition 1.12. Given any collection tv^ , tv ^ ,..., tv^ of permutation patterns (classical or general¬ 
ized), we denote by 


k k 

S n ( 7T (1) ,7T (2) ,... ,7T (fe) ) = P| S n ( 7T W ) = ^{a £ & n \ (7 avoids tt w } 

2=1 2—1 

the avoidance set of permutations a £ 6 n such that a simultaneously avoids each of the patterns tv^ , tv^ ,..., 
Tv( k \ Furthermore, the set 

P S'„(7T (1) ,7r (2) ,...,7r (fe) ) 
n> 1 

is called the (pattern) avoidance class with basis {7r(b, 7r( 2 ), ... ,7r^}. 


More information about permutation patterns in general can be found in [4]. 


2. Barred and Unbarred Generalized Pattern Avoidance 


An important further generalization of the notion of generalized permutation pattern requires that the 
context in which the occurrence of a generalized pattern occurs be taken into account. The resulting concept 
of barred permutation patterns , along with the accompanying notation, first arose within the study of stack- 
sortability of permutations by J. West m Given how naturally these barred patterns now arise in the 
study of Patience Sorting (as illustrated in both i5] and Section 0 below), we initiate their systematic study 
in this section. 
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Definition 2.1. A barred (generalized) permutation pattern (3 is a generalized permutation pattern in 
which overbars are used to indicate that barred values cannot occur at the barred positions. As before, we 
denote by ... ,(3 the set of all permutations a G © ra that simultaneously avoid (3^ l \ ... ,/3^ (i.e., 

permutations that contain no subsequence that is order-isomorphic to any of the (3^\ ■ ■ ■ ,(3^). 

Example 2.2. A permutation a = <J\cJn - ■ ■ a n G S n ^ £„ (3-5-2-4-1) contains an occurrence of the 
barred permutation pattern 3-5-2-4-1 if it contains an occurrence of the generalized pattern 3-2-4-1 (i.e., 
contains a subsequence (cr,;,, cq 2 ,..., cq m ) that is order-isomorphic to the classical pattern 3241) in which no 
value larger than the element playing the role of “4” is allowed to occur between the elements playing the 
roles of “3” and “2”. This is one of the two basis elements for the pattern avoidance class used to characterize 
the set of 2 -stack-sortable permutations mmm (The other pattern is 2-3-4-1, i.e., the classical pattern 
2341.) 

Despite the added complexity involved in avoiding barred permutation patterns, it is still sometimes 
possible to characterize the avoidance class for a barred permutation pattern in terms of an unbarred gen¬ 
eralized permutation pattern. The following theorem gives such a characterization for the pattern 3-1-42. 
(Note, though, that there is no equivalent characterization for such barred permutation patterns as 13-42 
and 3-5-2-4-1.) 

Theorem 2.3. Let B n = TT denote the n th Bell number. Then 

(1) S n { 3-1-42) = 5„(3-l-4-2) = S„(23-l) 

(2) |(3-1-42) | = B n 

Proof. (Sketch) 

As in [Jj, we see that each of these sets consists of permutations having the form 

a = <71(11(7202 • ■ • <7fcOfc, 

where a*, > Ofc_i > • • • > 02 > ai are the successive right-to-left minima of o (reversing the order of the 
elements in Definition oi) and where each segment < 7^(4 is a decreasing subsequence. □ 

Remark 2.4. We emphasize the following important consequences of Theorem l2~3l 

(1) Even though S'„(3-l-42) = 5' n (23-l) by Theorem 12.3( 11. it is more natural to use avoidance of 
the barred pattern 3-1-42 in studying Patience Sorting. As shown in (5] and elaborated upon in 
Section 0 below, S n (3-1-42) is the set of equivalence classes of & n modulo the transitive closure 
of the relation 3-1-42 ~ 3-1-24. (I.e., two permutations < 7 , r G & n are equivalent if the elements 
creating an occurrence of one of these patterns in <7 form an occurrence of the other pattern in r.) 
Moreover, each permutation a G & n in a given equivalence class has the same pile configuration 
R(cr) under Patience Sorting, a description of which is significantly more difficult to describe for 
occurrences of the unbarred generalized permutation pattern 23-1. 

(2) Marcus and Tardos proved in |14 | that the avoidance set S n ( 7r) for any classical pattern n grows at 
most exponentially fast as n —> 00 . (This was previously known as the Stanley-Wilf Conjecture.) 
The Bell numbers, though, satisfy \ogB n = n(logn — log log n + 0(1)) and so exhibit superex¬ 
ponential growth. (See nu for more information about Bell numbers.) While it was previously 
known that the Stanley-Wilf Conjecture does not extend to generalized permutation patterns (see, 
e.g., <7]), it took Theorem 12.31 2') (originally proven in [5J using Patience Sorting) to provide the 
first verification that one also cannot extend the Stanley-Wilf Conjecture to barred generalized 
permutation patterns. 

A further abstraction of barred permutation pattern avoidance (called Bruhat-restricted avoidance ) 
was recently given by A. Woo and A. Yong in US The result in Theorem 12. 31 21 has led A. Woo 
to conjecture to the second author that the Stanley-Wilf ex-Conjecture also does not extend to this 
new notion of pattern avoidance. 
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We conclude this section with a simple corollary to Theorem IP that gives similar equivalences and 
enumerations for some barred permutation patterns that also arise naturally in the study of Patience Sorting 
(see Proposition Id.21 and Theorem I3.fi! in Section 0 below). 

Corollary 2.5. Using the notation in Theorem ro 

(1) £ n (31-4-2) = £„(3-1-4-2) = S„(3-12) 

(2) £„ (2-41-3) = >S„(2-4-l-3) = S„(2-4-l-3) = 5„(2-41-3) 

(3) |S„(2-41-3)| = |S„(31-4-2)| = |5„(3-1-42)| = B n . 

Proof. (Sketches) 

(1) Take reverse complements in S' n (3-l-42) and apply Theorem l2.3l 

(2) Similar to (1). (Note that (2) is also proven in flj.) 

(3) This follows from the fact that the patterns 3-1-4-2 and 2-4-1-3 are inverses of each other. □ 

3. Patience Sorting under Restricted Input 

3.1. Patience Sorting on Restricted Permutations. The similarities between the Extended Pa¬ 
tience Sorting Algorithm (Algorithm 11.211 and RSK applied to permutations is perhaps most observable in 
the following simple proposition: 

Proposition 3.1. Let i k = 1-2- • • • -k = 12 • • • k and j fc = k-■ ■ ■ -2-1 = k ■ ■ ■ 21 be the classical monotone 
permutation patterns. Then there is 

( 1 ) a bijection between S n ( ifc+i) and pairs of pile configurations having the same composition shape 
7 = (71 > 72 ) • • • , 7 m) (= n but with at most k piles (i.e., m < k). 

(2) a bijection between S n (j k+1 ) and pairs of pile configurations having the same composition shape 
7 = ( 71 ,72, • • • ,7m) 1= n but with no pile having more than k cards in it (i.e., 7 j < k for each 
i = 1 , 2 ,..., m). 

Proof. (Sketches) 

(1) Given er £ & n , a bijection is formed in J2 between the set of piles R(cr) = { 77 , 7 - 2 ,... , 77 } formed 
under Patience Sorting and the components of a particular distinguished longest increasing subse¬ 
quence in a. Since avoiding the monotone pattern i k +} is equivalent to restricting the length of the 
longest increasing subsequence in a permutation, the result then follows. 

(2) Follows from (1) by reversing each of the permutations in S n (ik+i) in order to form S n (] k+1 ). □ 

Proposition EH states that Patience Sorting can be used to efficiently compute the length of both 
the longest increasing and longest decreasing subsequences in a given permutation. In particular, one can 
compute these lengths without needing to examine every subsequence of a permutation, just as with RSK. 
However, while both RSK and Patience Sorting can be used to implement this computation in 0(nlog(n)) 
time, an extension of this technique is given in 2] that also simultaneously tabulates all of the longest 
increasing or decreasing subsequences without incurring any additional asymptotic computational cost. 

As mentioned in Section El above, Patience Sorting also has immediate connections to certain barred 
permutation patterns: 

Proposition 3.2. 

(1) S 1 ,, (3-1-42) = {RPW(R(a)) \ a £ 6„}. In particular, given a £ S'„(3-l-42), the entries in each 
column of the insertion piles R(cr) (when read from bottom to top) occupy successive positions in 
the permutation a. 

(2) 5„(2-41-3) = {RPW(R{a~ 1 )) \ a £ ©„}. In particular, given a £ S n (2-4 1-3), the columns of the 
insertion piles R(cr) (when read from top to bottom) contain successive values. 

Proof. Part (1) is proven in {§j, and part (2) follows immediate by taking inverses in (1). □ 
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As an immediate corollary, we can characterize an important category of classical permutation patterns 
in terms of barred permutation patterns. 

Definition 3.3. Given a composition 7 = (71,72, ,7m) |= n, the (classical) layered permutation 

pattern 7r 7 £ & n is the permutation 

7i ’ (7i - 1) ’ ’ • 1 • (7i + 72 ) • (71 + 72 - 1) • • • (71 + 1)''' n ' (rc — 1) ’ ’' (7i + 72 H-h 7m —1 + 1). 

Example 3.4. Given 7 = (3,2,3) |= 8, the corresponding layered pattern is 71 ( 3 , 2 , 3 ) = 32154876 £ S 8 
(following the notation in Il5|h Moreover, applying Extended Patience Sorting (Algorithm 1 1.21) to 773 , 2 , 3 ): 

16 16 

^( 77 ( 3 , 2 , 3 )) = 2 4 7 and * 5 ( 773 , 2 , 3 )) =247 

3 5 8 3 5 8 

Note in particular that 773 , 2 , 3 ) satisfies both of the conditions in Proposition 13.21 which illustrates the 
following characterization of layered patterns: 

Corollary 3.5. S n (3-1 -42,2-41-3) is the set of layered patterns in & n . 

Proof. Apply Proposition ^. 2l noting that S n ( 3-1-42,2-41-3) = S n (23-1,31-2) (as considered in §J). □ 

As a consequence of this interaction between Patience Sorting and barred permutation patterns, we can 
now explicitly characterize those permutations for which the initial iteration of Geometric Patience Sorting 
(as defined in Section ITTI abovel yields non-crossing lattice paths. 

Theorem 3.6. The set «S rl (3T-42,31-4-2) consists of all reverse patience words having non-intersecting 
shadow diagrams. (I.e., no shadowlines cross in the 0 th iterate shadow diagram.) Moreover, given a permu¬ 
tation a £ S'„(3T-42, 31-4-2), the values in the bottom rows of R{a) and S(a) increase from left to right. 

Proof. From Theorem l2. 3l and Corollarv l2.5l R(S n (3-1-42,31-4-2)) = R(S n ( 23-1,3-12)) consists exactly 
of set partitions of [n] = {1,2, ...,n} whose components can be ordered so that both the minimal and 
maximal elements of the components simultaneously increase. (These are called strongly monotone partitions 

in JJj). 

Let a £ SVi(3-I-42, 31-4-2). Since a avoids 3-1-42, we have that a = RPW(R(a)) by Proposition 13.21 
Thus, the 7 th shadowline L(a) of a is the boundary of the union of shadows with generating points in 
decreasing segments (XiOq, i £ [k], where cqaj are as in the proof of Theorem l2.3l Let bi be the 7 th left-to-right 
maximum of <7. Then bi is the left-most (i.e. maximal) entry of cqai, so cqai = bicr'^ for some decreasing 
subsequence o[. Note that a[ may be empty so that bi = at. 

Since bi is the 7 th left-to-right maximum of a , it must be at the bottom of the 7 th column of R(cr) 
(similarly, m is at the top of the i th column). So the bottom rows of both R(a) and S(a) must be in 
increasing order. 

Now consider the 7 th and j th shadowlines L[°\a) and (a) of a, respectively, where i < j. We have 
that bi < bj from which the initial horizontal segment of the 7 th shadowline is lower than that of the j th 
shadowline. Moreover, cq is to the left of bj, so the remaining segment of the 7 th shadowline is completely 
to the left of the remaining segment of the j th shadowline. Thus, L,f\a) and L^\a) do not intersect. □ 

In [§) the authors actually give the following stronger result: 

Theorem 3.7. Each iterate Dgfy(cr) (m >0) of a £ & n is free from crossings if and only if every row 
in both R(a) and S(a) is monotone increasing from left to right. 

However, this only characterizes the output of the Extended Patience Sorting Algorithm involved. As such, 
Theorem iron provides the first step toward characterizing those permutations that result in non-crossing 
lattice paths under Geometric Patience Sorting. 
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We conclude this section by noting that, while the strongly monotone condition implied by simultaneously 
avoiding 3-1-42 and 31-4-2 is necessary to alleviate such crossings, it is clearly not sufficient. (The problem 
lies with what we call “polygonal crossings” in the shadow diagrams in jgj, which occur in permutations 
like <r = 45312.) Thus, to avoid crossings at all iterations of Geometric Patience Sorting, we need to 
impose further “ordinally increasing” conditions on the set partition associated to a given permutation 
under Patience Sorting. In particular, in addition to requiring just the minima and maxima elements in the 
set partition to increase as in the strongly monotone partitions encountered in the proof of Theorem ED 
it is necessary to require that every record value simultaneously increase under an appropriate ordering of 
the blocks. That is, under a single ordering of these blocks, we must simultaneously have that the largest 
elements in each block increase, then the next largest elements, then the next-next largest elements, and so 
on. E.g., the partition {{5,3,1}, {6,4, 2}} of the set [6] = {1, 2,..., 6} satisfies this condition. 

3.2. Invertibility of Patience Sorting. It is clear that the pile configurations corresponding to two 
permutations under the Patience Sorting Algorithm need not be distinct in general (e.g., R(3142) = i?.(3412)). 
As proven in |5], two permutations give rise to the same pile configuration under Patience Sorting if and 
only if they have the same left-to-right minima subsequences (e.g., 3142 and 3412 both have the left-to- 
right minima subsequences 31 and 42). In this section we characterize permutations having distinct pile 
configurations under Patience Sorting in terms of certain barred permutation patterns. We then establish a 
non-trivial enumeration for the resulting avoidance sets. 

Theorem 3.8. A pile confiquration pile R has a unique preimaqe a £ &„ under Patience Sortinq if and 
only ifae S n (3-1 -42, 3-1-24). 

Proof. (Sketch) 

It is clear that every pile configuration R has at least one preimage, namely its reverse patience word 
a = RPW(R). By Proposition 13.21 reverse patience words are exactly those permutations that avoid the 
barred pattern 3-1-42. Furthermore, as shown in jj^j, two permutations have the same insertion piles under 
Extended Patience Sorting (Algorithm II .211 if and only if one can be obtained from the other by a sequence 
of order-isomorphic exchanges 3-1-24 -w 3-1-42 or 3-1-42 3-1-24. (I.e., the occurrence of one pattern is 

reordered to form an occurrence of the other pattern.) Thus, it is easy to see that R has a unique preimage 
a if and only if a has no occurrence of 3-1-42 or 3-1-24. □ 


Given this pattern avoidance characterization of invertibility, we have the following recurrence relation 
for the number of permutations having distinct pile configurations under Patience Sorting: 


Lemma 3.9. Set f(n) = | S n (3 -1-42,3-1-24)| and, for k < n, 


f(n, k) = \{a£ (3-1-42,3-1-24) : a{l) = k}\. 


Then f{n) = Y%=i 

(3.1) 

(3.2) 

(3.3) 

(3.4) 


/(n, k), and we have the following recurrence relation for f(n , k): 

f(n, 0) = 0 for n > 
f{n,l) = f{n,n) =f(n- 1) for n> 
f(n, 2) = 0 for n > 

f(n,k) = f(n,k-l) + f(n-l,k-l) + f(n-2,k-2) for n> 


1 

1 

3 

3 


subject to the initial conditions /(0) = /(0,0) = 1. 


Proof. First note that Equation ED is the obvious boundary condition for k = 0. 

Now suppose that the first letter of a £ S n (3-1-42, 3-1-24) is cr(l) = 1 or n. Then er(l) cannot be part of 
any occurrence of 3-1-42 or 3-1-24 in a. Thus, deletion of <r( 1), and subtraction of 1 from each component 
if <t(1) = 1, yields a bijection with S n -\ (3-1-42, 3-1-24) so that Equation (13.211 follows. 

Similarly, suppose that the first letter of a £ S n (3-1-42,3-1-24) is cr(l) = 2. Then the first column of 
R(a) must be * regardless of where 1 occurs in a. Therefore, i?(cr) has a unique preimage a if and only if 
ct = 21£6 2 so that Equation (EH1) follows. 
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Finally, suppose that cr £ S n ( 3-1-42,3-1-24) with 3 < k < n. Since cr avoids 3-1-42, a is a RPW by 
Proposition HU and hence the left prefix of a from fc to 1 is a decreasing subsequence. Let cr' be the 
permutation obtained by interchanging the values k and k — 1 in cr. Then the only instances of the patterns 
3-1-42 and 3-1-24 in cr' must involve both k and k — 1. Note that the number of cr for which no instances of 
these patterns are created by interchanging k and fc — 1 is exactly /(n, k — 1). 

There are then two cases in which an instance of the barred pattern 3-1-42 or 3-1-24 will be created in 
cr' by this interchange: 

Case 1. If k — 1 occurs between cr(l) = k and 1 in cr, then cr(2) = fc — 1, so interchanging k and 
k — 1 creates an instance of the pattern 23-1 via the subsequence ( k — 1, k , 1) in cr'. Thus, by Theorem 12.31 
cr' contains 3-1-42 from which cr' £ S n (3-1-42) if and only if k — 1 occurs after 1 in cr. Note also that if 
cr(2) = k — 1, then deleting k yields a bijection with permutations in S n -\ (3-1-42, 3-1-24) that start with 
k — 1. So the number of permutations counted in Case 1 is exactly f(n— 1, k — 1). 

Case 2. If fc — 1 occurs to the right of 1 in cr, then cr' both contains the subsequence (fc — 1,1, fc) and 
avoids the pattern 3-1-42, so it must also contain the pattern 3-1-24. If an instance of 3-1-24 in cr' involves 
both fc — 1 and fc, then fc — 1 and fc must play the roles of “3” and “4”, respectively. If the value £ preceding 
fc is not 1, then the subsequence (fc — 1,1,£, fc) is an instance of 3-1-24, so (fc — 1,£, fc) is not an instance 
of 3-1-24. Therefore, for cr' to contain 3-1-24, fc must follow 1 in cr', and so fc — 1 follows 1 in cr. If the 
letter preceding 1 is some m < fc, then the subsequence (to, 1, fc — 1) is an instance of 3-1-24 in cr, which is 
impossible. Therefore, fc must precede 1 in cr, from which cr must start with the initial segment (fc, 1, fc — 1). 
But then deleting the values fc and 1 and then subtracting 1 from each component yields a bijection with 
permutations in S n -2 (3-1-42,3-1-24) that start with fc — 2. It follows that the number of permutations 
counted in Case 2 is then exactly f(n — 2, fc — 2), which yields Equation (13.411 . □ 


If we denote by 

oo n 

y) = E E /(«> k)x n y k 

n= 0 k=0 

the bivariate generating function for the sequence {f(n, k)} n >k> o, then Equation implies that 
(l-y-xy- x 2 y 2 )$(x, y) = 1 - y - xy + xy 2 - xy 2 $(xy, 1) + xy( 1 - y - xy)$(x, 1). 
Moreover, using the kernel method, one can show that 


where F{x) = J2 n > o f( n ) xn = 1) i s the generating function for the sequence {/(n)}„> 0 - 

We conclude with the following main enumerative result about invertibility of Patience Sorting. 
Theorem 3.10. Denote by F n the n th Fibonacci number (with Fq = F\ = 1) and by 

a(n, fc) — ^ ) F ni F n2 ... F nk 

n 1 ,...,n lc >0 

mH- \-rik=n—k —2 

convolved Fibonacci numbers for n > k + 2 (where a(n, fc) := 0 otherwise). Then, defining 


^ \/l + 2x + 5x 2 — x —l ( \J\ + 2x + 5a; 2 — x 

x + 1 -I---- F(x) - F - 


2x 
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|7(o)l 


' 1 ' 


0 

/( i) 


^0 


0 0 

m 


Fi 


a(2,0) 0 0 

/( 3 ) 

, F = 

f 2 

, and A = ( a ( n , k )) n ^> 0 = 

a(3,0) a(3,1) 0 0 

/( 4) 


f 3 


a(4,0) a(4,1) a(4,2) 0 0 







we have that X = (I — A) 1 F, where I is the infinite identity matrix and A is lower triangular. 
Proof. (Sketch) 

From Equations FTFl :FH1 . we can conjecture an equivalent recurrence where m and ifOll are replaced 
by the following equation (here 5 n k is the Kronecker delta function): 


k -3 

(3.5) f(n, k) = c(fc, m)f(n - k + m) + 5 n kF k _ 2 , n>k> 2. 

m=0 

For this relation to hold, the coefficients c(k , m) must satisfy the following recurrence relation: 

c{k , m ) = c{k — 1, to — 1) T c{k — 1, m) + c{k — 2, m), k > 2, 

or, equivalently, 

c(k — 1, m — 1) = c(fc, to) — c(fc — 1, to) — c{k — 2, to), k > 2, 

with c(2, 0) = 1 and c(fc, to) = 0 in the case that fc < 2, to < 0 or ?n > /c — 2. This implies that the generating 
function for the sequence {c(fc,TO)}fc>o (for each to > 0) is 

T m+2 

n>0 V y 

It follows that the coefficients c(k,m) = a(h,m) in Equation (|3.5f) are convolved Fibonacci numbers m 
forming the so-called skew Fibonacci-Pascal triangle in the matrix A = (a(k,m))k,m> o- I n particular, the 
sequence of nonzero entries in column m > 0 of A is the m th convolution of the sequence {i 7, rl } n >o. 
Combining the expansion of /(n,u) from Equation (15.511 with Equation (15.211 . we obtain 

n —2 

f(n ) = a(n, to)/(to) +F„_!, 

m=0 

which is equivalent to the matrix equation X = AX + F. Since I — A is clearly invertible, the result 
follows. □ 

Due to space restrictions, we omit a direct bijective proof of Theorem m that will be included in the 
full article. 

Remark 3.11. Note that A is a strictly lower triangular matrix with zero sub-diagonal. From this it 
follows that multiplication of a matrix B by A shifts the position of the highest nonzero diagonal in B down 
by two rows, so (I — A) -1 = ^ n>0 A" as a Neumann series, and thus all nonzero entries of (I — A)” 1 are 
positive integers. 


Finally, one can explicitly compute 
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r° 









' 1 








0 

0 








0 

1 







i 

0 

0 







1 

0 

1 






i 

1 

0 

0 






1 

1 

0 

1 





2 

2 

1 

0 

0 





3 

2 

1 

0 

1 




3 

5 

3 

1 

0 

0 



7 

6 

3 

1 

0 

1 



5 

10 

9 

4 

1 

0 

0 



21 

16 

10 

4 

1 

0 

1 


8 

20 

22 

14 

5 

1 

0 0 



66 

50 

30 

15 

5 

1 

0 1 




















from which the first few values of the sequence {f(n)}n >o are immediately calculable as 

1,1,2,4,9,23,66,209, 718,2645,10373,43090,188803,869191,4189511,21077302,110389321 
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